(19) 




JAPANESE PATENT OFFICE 



PATENT ABSTRACTS OF JAPAN 

(11) Publication number: 10194562 A 



(43) Date of publication of application: 28 . 07 . 98 



(51) Intel B65H 37/04 
B41J 21/00 
B42C 19/00 




(21) Application number: 08351479 

(22) Date of filing: 27 . 12 . 96 


(71) Applicant: 

(72) Inventor: 


MINOLTA CO LTD 

HIRANO AKIRA 
KOBAYASHI TADASHI 





(54) IMAGE FORMING DEVICE 



(57) Abstract: 

PROBLEM TO BE SOLVED: To provide an image 
forming device capable of inserting plural sheets in a 



special cover and binding them on the arbitrary 
right-hand or left-hand stitch. 

SOLUTION: An Image forming device comprises a 
copying machine 10 which reads in original images as 
data and can change the direction of the images 
outputted on the basis of the data, a stacker means 11b 
of holding plural sheets outputted from the copying 
machine 10 and stacked thereon, and a binder means 5 
of inserting the plural sheets from the stacker means 
lib in a special cover and binding them. The right-hand 
or left-hand stitch chosen in advance determines the 
direction of images outputted on the sheets. 
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Intelligere: Latin for 'xinderstand', 'appreciate'; 

from inter: 'between', 'among' and legere: 'to pick', 'to gather', 'to choose', 'to read'. 
Intelligent: Who can read between the lines. 

1 Introduction 

Some researchers have recently been calling attention to certain lasting "affor- 
dances" of paper documents over digital ones [21,17]. Whereas the relation- 
ship between the two media is often assumed to be one of competition, in fact 
it is one of complementarity. While digital documents are dynamic (evolving 
in time), immaterial (made of infonrnational substance), and agentive (capable of 
initiating actions), paper documents are: 

Permanent: once printed they are "frozen", provide a stable memory of a docu- 
ment historical state, can be destroyed but not altered; 

Portable: they are light, can be transported autonomously, can be transferred 
hand-to-hand in a social encounter; 

Contextualized: they are parts of the physical world, have a material richness 
and re-markability, their varied appearance and physical surroundings en- 
sure an abundant mnemonic context. 

Intelligent Paper is a proposal for bridging the gap between the physical 
world and the digital world. Intelligent Paper is made up of three elements: 

Support Each sheet of paper (physical page) is identified by a code page-id which 
characterizes it uniquely among all pages world-wide; 

Input device An input device, called a pointer, is provided. When the user posi- 
tions the pointer on the page and clicks it, the pointer's coordinates pointer- 
loc are recogiuzed; 

Communication infrastructure The pair <page-id, pointer-loc> is sent over the Web; 
the page-id is decoded as a network address, and the pointer-loc is inter- 
preted by a program at this address. This results in an appropriate output 
action on cm output peripheral close to the user. 
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Fig. 1. Intelligent Paper 



An illustration is given in Fig. 1. The user is looking at a paper map of Paris, 
indicating such places of interest as the Louvre or the Sacr6-Coeur. She posi- 
tions the pointer over the Louvre, and cUcks. The page-id is decoded by the com- 
munication infrastructure into a URL belonging to the publisher of the map. 
This publisher owns an electronic version of the map, which allows it to realize 
that the transmitted pointer-loc corresponds to the location of certain marks on 
the page, representing "Le Louvre". The publisher associates freely chosen ac- 
tions with these marks, such as sending a video presentation of ti:\e museum to 
a TV peripheral close-by. 

With Intelligent Paper, each piece of paper retains all conventional stationery 
quaUties. A book, a newspaper, a prospectus can still be used in the usual ways, 
without any special equipment. However, if the user owns a pointer, and is near 
a peripheral device connected to ihe Web, the paper document assumes the be- 
havior of a touch-sensitive screen. It becomes crunched with aibitrarily complex 
information, provided by the document's pubUsher at its web site. How is this 
achieved? 

Intelligent Paper pages are standard sheets of paper entirely covered with 
printed marks, invisible to the human eye, but visible to the pointer. These 
marks convey two types of information: uruque page identification and local 
coonlinates relative to the page frame (plus possibly an encryption code, see 
section 5). When the user positions the pointer device at a certain location on 
the page, the pointer reads these invisible marks and determines from them the 
uruque page identification and its own current location on the page, and then 
sends this informatiori for interpretation over the web. 
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I. The paper-pointer relatioixship is iderxtical to the usual screen-mouse rela- 

o p!!^^^h'vsical-Daee is tightly coupled to its digital-page counterpart; the 
™Uon bttweS L pair [physfcal-page, digital-page) being mamtamed 
:rsfa°bitr:;;:distances tt:^>ugh the globally unique pa.e.d and the ^- 
frastructure routing scheme. 

sSJon 4 sh^s "i^er appUcation examples. Section 5 introduces a few exten- 
sions and variants of the primary concept. 

2 An example: Reading Othello 

Paper by the "Uterature Qassics" publishing company. 
Here are some of the actions Ariane can carry out: 
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translation or a number of s'antiara uteraiy translation 

^^^^^^^ 

the Web page on "^^^/,*^f"^^ "^.^^5^^^^^^^ the Web page and 

write a smaU note to herseU: Lepanto . bne goes oaci^ r 

V » hr^v t INK TO WEB PAGE on the paper page. She then moves me 
selects the box link, wcd r j„ rlirks A liiAk is now regis- 

^^'^ Ihe battle of Up»to" will pop on tt,e sceen. 

underlying InteUigent Paper. 



3 Technology 




Pi,. A map of Europe P^^^^t ^^^^^^^^^ 

ftfnSfa^s^^S^^^^^^^ 

A blow-up of these marks is shown on^e '^'^^^^^^^'^l printed";^ con- 
have been used as the encodmg mechanism. The second layer p 
ventional coloured ihks and is visible to the user. 



397 



code layer The basic idea is illustrated in Fig. 2. DataGlyphs- is a ^^<^o\osy 

cellonthepagecanbeidentifiedby a pair of integers (..j), with 0 < ..j < ^ 

r ,^ for the ceU The pointer can use these numbers to identify the cell, ana « 
P':?rLTvWre.o"sie.l page (whe» „ .r. Hoat^g-poln. 
""'^'■^.^.o. Kas b«„ mad. , or ■'^^.'r.n^^lX?^ 

"""" 

identifying the digital-page counterpart of the physical page. 

«"'aTl i- .enns of m'-" l-8«>, whid, » cos.ly in .orms 
o,p~fse._o{p,«^^^ 

stnall number of bits exist which do not require the camerato see a hill cell, 
allow a maximal use of whatever area is captured by the camera [91. 
' We will see uses for the remaining bits in section 5. 
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more) steps: the router decodes some prefix of the page-id — this prefix does not 
need to be of fixed length, but the length can vary depending on the actual page- 
id decoded — and then passes the remaining bits to a second router, typically 
one at the publisher's site. This way of proceeding exploits the natural tendency 
of publisher to buy Intelligent Paper sheets in bulk, so that it may be known by 
the first router that a certain number of consecutive page-ids are "owned" by a 
certain publisher. 

Coding a page-id on paper is more efficient than coding full URLs directly; 
these need to be humetn- readable and nrmemonic, and so use more space than 
is necessary for unique identification: a URL can easily reach 40 characters in 
length, a compared to the 8 bytes needed for a 64 bit page-id; even more seri- 
ously, the length of the URL cannot be bounded a priori. 

Digital pages and the second indirection level The page-id and the router pro- 
vide a first indirection level when interpreting the user's pointing actions. Once 
the digital page has been identified thamks to this first indirection, a second level 
of indirection is provided by the pointer-loc7 

When the user clicks at a certain location (i , y) on the page, and sends these 
coordinates to the digital-page identified in the first step, the pair (x,y) may 
serve as an index into the actions associated with this digital page, whether 
they are output actions such as displaying information on the user's peripheral, 
content selecting actions (in a given system state, the user's pointing may be 
interpreted as "Start selecting text", in the next state as "End selecting text"), or 
any other kind. 

The mapping of positions to actions is the responsibility of the digital page 
and can be done in several ways. The problem is similar to the situation of 
interpreting mouse clicks in an hypertext document displayed on a conven- 
tional screen. Indeed, Intelligent Paper could be said to be an hypertext doc- 
ument displayed on a passive paper screen. It is therefore tempting to import 
as much of the existing GUI technology from the traditional "active display" 
world to the "passive display" world of Intelligent Paper. The approach that 
we are currently investigating is to use the Adobe Acrobat'^'^ suite of products 
[1]. In pcu-ticular. Acrobat Exchange permits one to display documents contain- 
ing hyperlinks on a screen. These documents can be produced from arbitrrary 
PostScript files cind have the same look and disposition on the screen as the 
printed PostScript documents have on paper. Because of this isomorphism be- 
tween the screen eind the paper version, it is relatively simple to map a position 
of the paper document into a position on the screen. Pointing actions on the pa- 
per can therefore be simulated by virtual mouse actions on the display; every 
action that the user could have performed on the screen, he can perform on the 
paper using this simulation. 

Pointer Several technologies exist for the pointer's camera. They range from 
high-speed, high-resolution, CCD cameras to low-end hand-held scamners. The 



^ A similar idea to these two indirection levels appears in [20]. 
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cost of these devices is constantly decreasing. Also decreasing is the cost of 
wireless telephony, so that it will shortly become reaUstic to tether the pointer to 
an Internet-enabled phone; it may even be possible in the near future to include 
wireless commurucations capabilities inside the pointer 

Visible and invisible inks In the description so far, we have assumed that the 
code-layer is printed using invisible ii\k. There is a technological chaUenge here. 
Several solutions exist; some of which use infrared inks, other ultra-violet ones. 
Also, in the scenario we have been describing, the code layer is printed under 
the visible ink layer; it is also possible to print the invisible ii\k layer over the 
visible one. Each choice has advantages and disadvantages. Printing the invis- 
ible ink layer under the visible one permits to distribute white sheets of Intel- 
ligent Paper which can then be printed (or written above) in visible inks in the 
traditional maimer, but there is the challenge of recovering the invisible code 
through the layer of visible inks. These issues are related to questions about the 
light-absorption spectra of both visible and invisible inks. 

Color printing uses three basic colored inks: cyan, magenta and yellow (some- 
times black ink is also used but we will ignore that here). These inks work 
as transludd fUters which each eliminate a certain range of frequencies in the 
white Ught spectrum. By so-caUed "subtractive" combination of colors, an arbi- 
trary color can be reproduced by superposing the three basic colors in prx^per 
quantities [4]. In an idealized situation, each ink removes a strictly delimited in- 
terval of visible range frequencies and leaves untouched the other frequencies. 
One can in principle add a new "invisible" ir^c layer which removes frequencies 
only in a specific interval of invisible frequencies. This layer does not, in prin- 
ciple, interfere with the three visible layers, whether it is printed under or over 
them, and it can be detected by a camera equipped with an adequate optical 
filter. In practice, most commercial colour inks do not have a perfect absorption 
spectrum, and finding the right chemical composition for invisible ir\ks is an 
active research area [5, 6]. 

If one does not wish to use invisible ink, partial implementations of Intelli- 
gent Paper are possible. Because visible DataGlyphs are miimnally obstructive 
visually, they can be used as a gray background in otherwise white areas of pa- 
per. In this way, at least some areas of the paper document can be made active 
(for instance areas under words in ruru\ing text), and many of the capabilities 
of Intelligent Paper retained. 



4 Some Applications 
Getting information 

Maps A map of Paris can provide electroruc information about public trans- 
portation; by selecting two different points on the map, the user can be told 
about the most efficient bus and subway routes. Events at a certain location, 
such as the Op^ra, can be updated in real time on the Web site, though the user 
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Still uses the same old physical copy of the map. One single, static, paper docu- 
ment becomes dynamic by being linked to constantly updated digital data. 

Pocket encyclopedia A large encyclopedia can be condensed into a pocket 
version, giving concise descriptions, while the complete version is accessed 
through the Web or on a CD. A large dictionary can be produced on paper in a 
reduced format, while retaining the capabilities of the full version, and having 
extended functionalities such as thesaurus searches. 

Periodicals A travel magazine can be augmented with touristic information 
about places described. A music joumal can be ermched with recordings de- 
Uvered either through the Web or on a CD distributed with the magazine. A 
scientific joumal can have Uve bibUographic lii\ks permitting direct access to 
the text of cited articles. A professional newsletter can allow access to financial 
databcises. 

Daily papers In this age of immediate ir\fonnation, dailies have difficulties 
competing with media such as TV and radio. By being printed on Intelligent 
Paper, they can retain their specific advantages — journalistic outlook, in-depth 
analysis — while allowing ii\stant ir\formation updates. Pointing on the head- 
line for 'Trench socialists estimated to win by narrow margin", the reader can 
have ir\stant access to the latest polls, which can be deUvered to her on a TV 
screen, on a computer, or through the telephone. Other functionalities may in- 
clude pointing at a name to fincJ'aU its mentions in recent issues or to display a 
biography navigating to related recent articles, or selecting a TV broadcast by 
clicking on its entry in a program listing. 

Active personal notebooks A smaU personal address book may contain in 
handwritten form frequently accessed ir\formation, such as name or telephone 
number, while less frequently accessed information, such as street address, pho- 
tograph, or personal notes, are relegated to digital form. Personal handwritten 
annotations in a notebook can link to web sites, to personal documents, or even 
be associated with actions such as "start the coffee machine". 

Meeting agenda InteUigent Paper sheets with a printed agenda can be dis- 
tributed to participants in a meeting. These sheets can be used later by them 
to access recorded minutes of the agenda items, to find information about other 
meeting attendants, or to cor\sult status of follow-up actions. 



Interacting with services 

Hotel reseroations They can be made by accessing hotel sites from a tourist 
guide or a magazine. When the hotel page appears, the user may enter tex- 
tual infonnation (his name, address, etc.) through a small virtual keyboard on 
the scr^n. Alternatively, this keyboard can be printed directly on the physical 
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^ fi>w extensions and vananis u 

device consists in ptovidmB « j^,e ^,elUget« P>P»' real-time. 
cS, men "-'X U ™nito,«i by the PO^« Xi. pKysl"' 

somesisnificantpolentulapp „„ He captured in realtime. 

. „a„d-«Ht.enno.es.a.end^S -^'2,::;ped, L ^y^^^^^l 

rrSt^rnt'^^e wdting ^-Xlt'irer- St.* as events ^ 
- rS^--^-^2>dSronaa„.ttVpe^P-- 

Xhis could also ue 
neaiby. 

., ^,„.y.o,decla,inspeHpKe„ls.,-S— Se--^' 

« See secHon 5. for other ways o 
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Encryption Codes and Digital Commerce Another noteworthy extension of In 
telligent Paper is obtained if the bits remaining, after pointer-loc and page-id dai ^ 
have been taken care of, are used for encryption purposes. The approach is th^ 
foUowing. Suppose that at pubUcation time, the publisher of an InteUigent Pa^ 
per page associates with each individual page-id a certain string of, say, 50 bits 
This encryption string encrypt-string is completely random and has no'connec" 
tion whatsoever with the page-id itself. It is printed on the code layer along with 
the page-id and pointer-loc codes. The relationship between the page-id and the 
encrypt-string is known only to the publisher. 

The encrypt-string has appUcations to controlling access rights and to digi- 
tal commerce. An example will suffice to give a gist of the approach. Consicler 
the case of a recording company. Digital Grammophon, which distributes mu- 
sic recordings over the Net, for a certain fee. Let's now imagine the following 
scenario. Digital Grammophon sells, through recoixl or book-shops, "listening 
cards" printed on Intelligent Paper and wrapped in a protecting envelope. One 
such card, for instance, lists the last sbc Mozart symphonies, in Bruno Walter's 
famous interpretation. Each movement is listed as a subheading under the cor- 
responding symphony heading. The customer buys the card, goes back home, 
cUcks on the heading for the first movement of Mozart's 40th, and the music is' 
played for him on his Intemet^nabled hi-fi system. 

Digital Grammophon wants to prevent illegal use of the card. Each card has 
a unique, distinct, page-id and this abneady affords some protection to the com- 
pany For instance, if the card was copied by some means. Digital Grammophon 
could prevent the simultaneous use of the same page-id from two different 
users.9 But the scheme is not completely air-tight. A skillful pirate, rather than 
simply copying the card, could try to discover how many "Bruno Walter's 
Mozart Symphonies" cards are sold by Digital Grammophon and what are their 
page-ids (in all probabihty they would be consecutive numbers). 

But, if, along with the page-id, the encrypt-string is coded on the card. Digital . 
Grammophon can tell whether the encrypt-string is the right one for this page-id. 
Nobody else can guess the correct encrypt-string unless one is prepared to try 
the 2^° (roughly orie million billion) possible combinations. Thus, the encryp- 
tion code guarantees that although copying is possible Jorging is not: the only way . 
to obtain a record card is by duplicating an existing one, not by guessing a new 
one. It is, however, in the interest of the buyer of a cand not to have it copied by 
other parties, unless it is for making a personal backup, which is acceptable, for 
very soon the countermeasures taken by Digital Grammophon would make his 
own card unusable. 

Having discussed two extensions of the basic concept, we will now consider 
two variants of it which are reitheT simplifications. 



' Several other schemes exist for controlling rights through the page-id. For instance, a 
given card could entitle its purchaser to a limited cumulative listening Hme. 



403 



f^on-Positional Intelligent Paper We have mentioned above that invisible ink 
poses certain technological challenges, and that visible DataGlyphs could be 
used to provide some of the capabilities of Intelligent Paper. In all these cases, 
however, we have been considering a "positional" version of Intelligent Paper! 
that is, one in which the second level of indirection, from pointing events to 
actions defined relative to a digital page, is done via transmitting the (x, y) co- 
ordinates of the pointer relative to the physical page. 

Another possibility exists. Suppose that, in a given multipage document, 
one has determined a certain number of "active areas", rectangular regions 
around words or icons in the document. Suppose now that the document is 
given a unique document identifier, which we write doc-id, and is similar to the 
page-id previously discussed.^^ Suppose also that each active area in the docu- 
ment is given a small numeric identifier num-id relative to this document. If a 
document contains no more than 1000 active regions, 10 bits are sufficient to 
uruquely identify each such region. 

Now, it is possible to print, in each active region, a visible DataGlyph patch 
— which may be partially occluded by words or icons — that encodes the num- 
id for that region." These patches, although visible, are visually non-obstructive 
and serve as a reminder to the user that a certain region is active. Suppose also 
that the doc-id is printed, again using visible DataGlyphs, in a cell at the bottom 
of each page (or possibly orUy,once on the cover page of the document). 

The user can then perform the following pointing actions. First he points to 
the doc-id at the bottom of the page to identify the digital-page (or rather the 
"digital-document") associated with the document. Then he points at some ac- 
tive region in the document. The p'ointer decodes the num-id of this region and 
transmits it to the digital page for interpretation, as in the scenarios discussed 
so far.*^ 

Although this scenario loses several of the advantages of full positional In- 
telligent Paper — fully simulating a mouse, including ability to point to pr^se 
locations, to select a text or a graphical object or to draw — it still retair\s several 
important capabilities and is easier to implement in the short-term. 

Intelligent Confetti and establishing links between Internet-embedded objects 
Intelligent Confetti are a further simplification of Intelligent Paper. They are 
small round stickers carrying a uruque page-id code and nothing else. The glob- 
ally unique page-id is transmitted to the page-id router which determines the ad- 
dress of the digital page associated with the corifetti. In this way each cor\fetti 
is globally associated with a unique digital document through which an action 
associated with the cor\fetti can be performed. The crucial difference between 



In fact, even in the case of positional Inlelligeht Paper, it may be useful to define doc-ids 
for multipage documents. 

DataGlyphs, in some of their variants, can accept a relatively high-level of occlusion 

" The first step — pointing to the doc-id — need not be repeated each time, as long as 
the user is pointing in the same document- 
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confetti and conventional barcodes lies in the scheme for assigning a globally 
unique page-id to each confetti — this numbering scheme being under the con- 
trol of the authorized Intelligent Paper producer — and for routing this page-id 
to its corresponding digital page in a universal way. Confetti cein be used in 
several ways: 

- Multimedia links can be added to conventional documents. By sticking a 
confetti in the margin of a conventional document, the user can associate 
a certain action with this confetti. The confetti producer provides the user 
with an interface on her computer through which she can define a link or 
an action to be associated with the confetti (for instance, ringing a phone 
number or playing a vocal annotation). When she later, from an arbitreuy 
location, peruses the document again and points on the confetti, this action 
is performed for her. 

- Confetti can be stuck on physical objects as well as paper documents. A 
confetti stuck on a mechanical part can later be used to display information 
about this part. ' 

One of the most important uses of confetti is in establishing lir\ks between 
Internet-embedded objects. Until now, we have finessed the issue of how the 
"output peripheral close to the user" (see section 1) is actually identified to 
the Intelligent Paper coirunimication infrastructure. One possibility is to as- 
sociate the peripheral with the pointer through a conventional computer di- 
alogue, where the pointer's owner is asked to identify a certain communication 
port at a certedn Internet address, if the peripheral is a computer display or a 
printer, or to enter a certain telephone number if the peripheral is a telephone. 
With confetti, this operation cein be done much more transparently: the user sim- 
ply clicks on a confetti stuck on the peripheral. The action associated to this confetti 
consists in establishing the peripheral to the communication infrastructure as 
being the current peripheral, and in transmitting further information about the 
peripheral characteristics permitting the output action to be performed or opti- 
mized. 

6 Conclusion 

We have discussed some of the potential applications of Intelligent Paper, and 
presented the underlying technology in its broad outline. Some aspects of this 
technology — invisible ink under visible ink, writer-pointer — are not fully 
mature yet, but progress is being rapidly made. We often take for granted, and 
do not not even pause to consider, many technologies — color printing, minia- 
turized cameras, wireless telephony — which would be considered to be sheer 
science-fiction if it were not for the disturbing fact that they already exist. 

Intelligent Paper is a proposal for making physical documents collaborate 
closely with digital documents. To take one instcirice where the tension be- 
tween the two worlds is evident, several authors [16, 18] have opposed the 
practices of writing, publishing and reading books to their counterparts on the 
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Web. Whereas strengths of the book are: linearity, concentration of informa- 
tion, clear borderline between internal and external, uniform viewpoint main- 
tciined throughout, minimization of redundancy, careful choice of content, and 
publisher's reputation, those of the Web are: multidimensional connectedness, 
freedom of content, open-endedness, variety of viewpoint, unlimited content, 
and absence of juridiction over quality. 

The very linearity of books imposes a lot of work on their designers. This 
effort is not simply an artifact of the book form, but an essential ingredient in 
the final result, which is orgaruzed for maximally efficient commuiucation with 
the reader. The Web is more anarchic, more free, but the user needs to work out 
her ovm "design" for her reading. 

Intelligent Books — books printed on Intelligent Paper — allow a comple- 
mentarity between the two levels. The printed document corresponds to in- 
formation which is highly saHent, "obligatory" and controlled, the digital doc- 
ument and its lir\ks to information which is "on demand", open-ended, and 
interactive. 

Whatever the current hype about Virtual Reality, the physiccd world is not 
going to disappear any time soon. The future of the human community is not 
about eliminating the feels and tastes of direct interaction with each other, places 
and objects, but about enriching this interaction. A world in which music stores, 
newsstands and bookshops have vanished would be a poorer world. A middle 
road will have to be found between digital information, dynamic, decentralized 
and open-ended but also somewhat cold and depersonalized and, on the other 
hand, traditional means of designing, producing and selling this ii\formation. 
We believe that Intelligent Paper represents one step towards such a balance. 
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